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1.  Introduction. 

Let  fXjj  ^  be  a  q-variate  (1  £  q  <  “>)  weakly  stationary  stochastic  process 
with  the  spectral  density  matrix  £.  In  the  Kolmogorov-Wiener  theory  of  prediction, 
one  assumes  that  the  spectral  density  f  and  the  Infinite  past  m£0}  are 

known  and  the  problem  Is  to  express  the  linear  least  squares  predictor  of  Xj^, 
denoted  by  Xj^.  In  terms  of  the  components  of  f  and  P.  It  can  be  shown  that  Xi^ 
can  formally  be  written  as 

Xj  ~  ^  An^_m, 

m“0 

where  Am » .  for  the  notation  see  Section  2. 

In  practice,  however,  one  seldom  knows  f  and  P,  and  therefore  It  Is  necessary 

A>. 

to  approximate  Xj^  by  some  properly  chosen  approxlmants  based  upon  the  available 
information.  Very  often,  this  available  Information  is  a  finite  segment  of  the 
pest,  l.e.  Here,  we  consider  the  ss  yet  imprscticsl  case  of 

knowing  f  and  Pj^,  and  this  will  provide  some  useful  results  for  the  case  when 
only  Pji  Is  known  and  f  has  to  be  estimated. 

Now,  knowing  f  and  P^  there  are  two  natural  approxlmants  for  Xp  The  first 

^  -  n-1 

one  being  the  partial  sum  of  the  series  representation  of  X.:X.  i  A^X-m* 

m-0 

The  second  approxlmant  Is  the  linear  least  squares  predictor  of  X]^  based  on 

n-1 

Pji,  denoting  this  by  we  have  X*^jj»  ^  AnmX_jj,  where  the  coefficients  Amn, 

’  tn“0 

0<m<n-l,  can  be  obtained  as  the  result  of  solving  a  minimization  problem.  Having 
Identified  these  two  approxlmants.  It  is  natural  to  ask  as  how  good  they  are. 
Namely  it  is  of  Interest  to  know  how  fast  ll^l”^l,nll  approach 

zero  as  n-*".  The  main  purpose  of  this  paper  is  to  find  rates  of  convergence  to 
zero  for  these  quantities,  or  the  errors  of  approximating  the  infinite  pre- 

Ak  ^  it 

dlctor  Xi  by  finite  predictors  Xj^jj  and 
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It  is  obvious  that  if  the  series  representation  of  Xj^  does  not  converge  in 

00 

the  mean,  then  llx  -X,  ll  ■  li  7  AX  ij  does  not  go  to  zero  at  all  as  n-+<».  This 
^  m-nfl  “  ■“ 

may  suggest  the  type  of  (smoothness)  conditions  which  one  may  need  to  impose  on 
{X  }  or  f  in  order  to  obtain  a  fast  rate  of  convergence. 

00 

The  rate  of  convergence  of  ||X.-X*  ||  is  dominated  by  that  of  ||  2 

and  ||Xj^  ^-X*  ^l|  ,  cf.  (2.9).  The  rate  of  convergence  of  ijXj^  ^-X*  ^||  is  expected 

to  play  fundamer.cal  roles  in  some  statistical  problens  related  to  the  estimation 
of  f,  when  only  is  available.  For  the  univariate  processes  this  Is  dem¬ 
onstrated  in  [2,6],  see  also  Section  4. 

Let  G  be  the  qxq  matrix  of  one-step  ahead  prediction  error.  Define  cr«tri7, 
aQ-(trfl^)^  and  lIXj-X*  ^|1  Note  that  for  q-1,  o-Oq.  With  this  notation 
the  outline  of  the  paper  is  as  follows.  Section  2  is  devoted  to  introducing 
some  preliminary  results  establishing  some  useful  relationships  among  <Jn,o  and 
OQ. 

For  a  univariate  stationary  process  an  important  result  of  Grenander  and 
Rosenblatt  [4,  Thm.4],  [5,  p.  188]  gives  necessary  and  sufficient  conditions  on  f 
for  the  rate  of  convergence  of  cf^(i-ao(-an-a)  to  decrease  at  least  exponentially  to 
zero.  The  rate  of  convergence  of  Ojj-Oq  plays  an  important  role  in  finding  the 
rate  of  convergence  for  finite  linear  predictors.  The  entire  Selctlon  3  is  devoted 
to  finding  an  appropriate  multivariate  extension  of  this  result  of  Grenander  and 
Rosenblatt,  l.e.  finding  necessary  and  sufficient  conditions  on  the  matrix  f 
such  that  6n“9  ^iT^q  decreases  to  zero  at  least  exponentially,  cf.  Theorems  3.2 
and  3.7.  The  proof  presented  in  [4,5]  depends  heavily  on  the  analytical  properties 
of  orthogonal  polynomials  with  respect  to  f.  Since  matrlclal  orthogonal  poly¬ 
nomials  may  not  enjoy  all  of  these  properties  or  they  are  hard  to  prove,  special 
attempt  is  made  to  prove  the  results  of  Section  3  without  adherence  to  the 


orthogonal  polynomials.  This  is  done  by  adapting  a  multivariate  version  of  a 
method  used  in  Devlnatz  [3],  cf.  Theorem  3.1.  Devinatz's  proof  of  the  necessity 
part  of  the  Result  of  Grenander  and  Rosenblatt  is  still  based  on  properties  of 
orthogonal  polynomials.  We  get  around  this  by  using  a  lemma  due  to  Mlamee  [9], 
cf.  Theorems  3.6  and  3.7. 

In  Section  4  under  the  boundedness  condition  on  the  spectral  density,  cf.  (4.1), 
we  show  that  the  approximation  errors;  ||  ,  Ijx^-X*  ^|1  and  llX^^jj-XJ^^H 

have  the  same  order  as  6n"9  ®n~^0*  Therefore,  combining  this  with  the  multivariate 
extension  of  the  result  of  Grenander  and  Rosenblatt  we  get  exponential  rate  of 
convergence  to  zero  for  these  errors  provided  that  the  entries  of  f  are  analytic 
functions  on  (-77,7:],  cf.  Theorems  4.1  and  4.2.  Theorem  4.3  provides  a  rate  of 
convergence  for  the  finite  linear  interpolator  of  {  Xn)  when  only  one  value,  say, 

Xq  is  unknown. 

2.  Preliminaries 

2 

Let  (Jl.F.P)  be  a  probability  space  and  A#«Lo(5),F»P)  the  Hilbert  space  of  all 
complex-valued  random  variables  on  SI  with  zero  expectation  and  finite  variance. 

The  inner  product  in  M  is  given  by 
(x,y)»Exy  ,  x,yeM. 

Following  [8]  for  an  integer  <^>1,  denotes  the  cartesian  product  of  M  with  itself 
q  times,  i.e.  the  set  of  all  column  vectors  X«(Xj,X2, • • • »Xq)  with  x^  EM,  i«l,2, . . . ,q. 

is  endowed  with  a  Gramlan  structure.  For  X  and  Y  in  their  Gramlan  is  defined 
to  be  the  qxq  matrix  (X,Y)  ■  [(x.,y.)]^  .  It  is  well-known  that  is  a  Hilbert 

^  J  i.J-1  q 

space  under  the  inner  product  ((X,Y))  -  trace  (X,Y)  ■  I  (x. ,y.)  and  norm 

J-1  ^  ^ 

II  X||  ■  A (X,X)),  provided  the  linear  combinations  are  formed  with  constant  matrices 
as  their  coefficients. 

Throughout  this  paper,  for  a  qxq  matrix  A-(a. .),  trA-  >  a..,  A*-(a..),  det  A 
stands  for  the  determinant  of  A,  A  ^  for  the  inverse  of  A  when  it  exists  and  HaI!  ^ 
for  the  Eucleadian  norm  of  A  i.e.  ||a||  g“(tr  AA*)^.  For  two  qxq  matrices  A  and  B, 
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A^B  means  that  A-B^O,  l.e.  A-B  Is  a  non-negative  definite  matrix.  The  qxq 
identity  matrix  is  denoted  by  I.  Functions  will  be  defined  on  [-7r>7r]  and  we 
identify  this  interval  with  the  unit  circle  in  the  complex  plane  in  the  natural 
way.  Values  of  a  function  f  defined  on  the  unit  circle  will  be  denoted  by  f(6) 

J  Qk 

instead  of  f(e^”).  dm  stands  for  the  normalized  Lebesgue  measure  on  [-Trtir]  •  i.e. 

dm(9)-d0/2Tr.  For  l^(h*’)  denotes  the  usual  Lebesgue  (Hardy)  space  of 

functions  on  the  unit  circle.  (H^  .  denotes  the  space  of  all  qxq  matrix- 

valued  functions  whose  entries  are  in  L^(H^). 

In  the  following  we  Introduce  a  few  concepts  which  are  needed  In  this  study. 

A  more  complete  discussion  with  proofs  can  be  found  in  [8, 15]. 

Let  {  Xq;  ncZ}  C  be  a  q-varlate  weakly  stationary  stochastic  process 

(WSSP)  with  the  spectral  density  matrix  f .  The  time  domain  of  {  X^}  is  defined 

by  sp  {  Xj^:  nc  Z}  (X),  where  sp  {  }  stands  for  the  closed  linear  span  of  elements 

of  {  }  In  the  metric  of  M^.  The  spectral  domain  corresponding  to  the  spectral 

2  2 

density  matrix  f  Is  denoted  by  L  (f)  and  is  defined  by  L  (f)  "  {  $;<!>  is  a  qxq  matrix 

valued  function  with  ||$11  -  Itv^  (9)f  (0)  <I>  *(0)dm(0)  <  «»}.  It  is  known  that 

2  ^ 

L  (f)  with  inner  product  given  by  (($,'1'))^“  /tr  $  f 'F  *dm(0)  is  a  Hilbert  space 
and  It  Is  Isomorphic  to  M(X) . 

The  problem  of  linear  least  squares  prediction  of  a  q-varlate  WSSF  {  X^j}  can 

be  stated  as  the  problem  of  finding  the  qxq  matrices  A^  such  that  the  linear 

least  squares  predictor  of  Xj^  based  on  Xgi»  m^0>  denoted  by  X^,  can  be  written  as 
/N  n 

X^sl.l.m.  ^  AinX_^,  where  1.1. m.  stands  for  the  convergence  In  the  norm  of 


(or  in  the  mean).  Let  d;»(Xi-Xi,Xi-Xj)  be  the  one-step  ahead  prediction  error 

matrix.  The  process  {  X„}  Is  said  to  be  of  full-rank  If  the  matrix  G  is  nonsingular. 

From  now  on,  we  assume  that  the  process  {  Xn)  is  purely  non-determlnlstlc  and 

full-rank.  This  is  equivalent  to  assuming  that  {  X^)  has  a  spectral  density  f 
1  2 

with  log  det  f  e  L  and  where  <I>cH  Is  an  outer  function  with  constant 

'qxq 

term  <I>q  We  refer  to  ^  as  the  generating  function  of  the  process  {  X^} . 


» 


» 


» 


Since  {  X^^}  Is  full-rank,  we  can  define  a  process  {  called  the  normalized 


■1 


Innovation  process  of  {  X„}  such  that  (Yn.Yn)  “  ^in.n^’  ®y  using  the  Wold's 
decomposition  [8]  we  have 


*n“  I  ^^n-m  » 

m^O 

(2.1) 

♦(e)- 1  cy"® 

m-O  “ 

Since  $  is  analytic  with  no  zeros  in  the  open  unit  disc,  hence  ^  ^  is  analytic 
and  denoting  its  Fourier  (Taylor)  coefficients  by  Dj^,  k-0,1,2,...,  we  have 

(2.2)  $‘^0)-^Dy“®. 

m“0 

with  this  notation  the  linear  least  squares  predictor  of  X]^  can  formally  be 
written  as 

(2.3)  Xl-'-Co  iDmflX-m  » 

m«0 

where  the  infinite  series  on  the  right  hand  side  of  (2.3)  may  not  converge  in 
the  norm  of  M^. 

We  say  that  the  linear  least  squares  predictor  of  Xi  has  a  mean  convergent 
autoregressive  representation  if  the  series  in  (2.3)  converges  in  the  norm  of  M** 
In  practice,  however,  we  have  only  a  finite  number  of  observations  XQ,X_p.. 
X_(n_i)  from  which  one  has  to  construct  a  linear  predictor  for  Xj.  For  this  one 
has  two  options.  The  first  one  being  the  partial  sum  of  the  series  in  (2.3).  T 
with  ^  and  denoting  this  linear  predictor  by  Xj^  ^  we  have 

- 

(2.4)  Xj^  Y  AnX-n 

’  m"0 

If  X]^  has  a  mean  convergent  autoregressive  representation,  then  the  norm  of  the 
error  of  approximating  X,  by  X.  is  given  by 

00 

(2.5)  1|X.-X  11-11  I  IhO  as  n-^. 

'■  m-rrt-1 

A  different  linear  predictor  for  X,,  denoted  by  X*  ,  can  be  obtained  as  the 
linear  least  squares  predictor  of  X.  based  on  n  observations  X  ,  O^m^n-1,  from 
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the  past.  Thus,  we  have 

n-1 

(2.6)  X*  -  I  . 

’  m-O 

n-1  2 

where  CKnKn-l,  can  be  found  by  minimizing  the  norm||Xi-  ^  BnPC_ni  ll  “ 

n  ‘imfi  imA  m"0 

/tr(I-  2.  f(I“  I  Bme^“”)*  dm(6)  over  all  choices  of  the  constant 

m»l  m-1 

matrices  Bm- 

Let  us  denote  the  minimum  value  of  this  norm  by  a  .  Then,  it  can  be  shown 

n 

that  this  minimum  Is  attained  for  the  polynomial  P  (6)«o  V  (6),  where 

r  i  m0  ' 

1^(0)“  1  U  e*"  is  the  unique  polynomial  of  degree  n  satisfying 

m»0 

(2.7)  /tr  IT  (6)f(0)  e“^*‘®dm(e)-6.  ,  0<k<n. 

This  unique  minimizing  polynomial plays  an  Important  role  In  this  paper. 
In  the  following  we  give  a  different  characterization  of  the  matrix-valued 
polynomial  17^. 

2 

Lenma  2.1.  Let  f-$$*  and  aQ*(tr  4q)  .  Then, 


(a)  min  /tr(I-P$)(I-P'l>)*dm»  dm. 


where  the  min  Is  taken  over  all  nth  degree  polynomials 
_im0 


P(9)> 


n 

y  P  e- 
nm 

m«0 


Proof,  (a)  Let  M^“sp{e^^®  4>;  0<^k<n>  In  L^(f).  Then  the  minimum  in  (a)  is 
attained  for  some  H  in  Kn,  if  and  only  if  I-H$  is  orthogonal  to  Mn  or  equivalently 

/tr(I-H(l))<l'*e"^'^®dm-0,  0<k<n. 

It  is  easy  to  show  that  l/j,  satisfies  the  above  relation,  (b)  follows  from 

direct  compulation  and  the  fact  that  /tr  $  dm«*^/an  (2.7)  with  k«0. 
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Q.E.D. 

Since  is  the  isomorph  of  Yq  (the  normalized  innovation)  the  following 

corollary,  which  follows  immediately  from  Lemma  2.1,  provides  a  different 

characterization  of  U  in  that  is  the  isomorph  of  the  linear  least 

n  On 

squares  estimate  of  Yq  based  on  the  finite  past.  Note  that  with  this  inter¬ 
pretation,  q-  is  the  norm  of  the  error  of  this  estimator. 

Corollary  2.2.  Under  the  notation  of  Lemma  2.1,  we  have 

min  /tr($  ^-*^P)f(4>  ^-v5^)dm“/tr('Ii  ^-v^^)f($  ^-»'Cf^V)*dm 

-q-Oo/On. 

It  turns  out  that  the  rate  of  convergence  of  q-  as  n-»<»  plays  a  crucial 

role  in  studying  the  rate  of  convergence  of  finite  linear  predictors  and  inter¬ 
polator  of  a  q-varlate  WSSP  This  is  shown  in  the  next  two  sections.  In 

the  following  we  record  several  inequalities  which  are  useful  in  connecting 
rates  of  convergence  of  different  quantities  related  to  the  problem  studied  here. 

/s  /w 

The  following  chain  of  inequalities  follows  from  the  definition  of  Xj,  Xj^ 


and  Xf 

l.n 


(2.8) 


l|Xl-Xi||  <  llXi-X*^^||  <  l|Xi-Xi,nl 


IIIX1-X1II+  II  I  A„X_„||  . 
m«n+l 

/s 

For  the  last  inequality  to  hold  it  is  necessary  to  assume  that  Xj  has  a  mean 
convergent  autoregressive  representation  and  this  will  be  assumed  throughout 
this  paper.  Also,  for  the  norm  of  the  error  of  approximation  of  X^  by  X#  we 

4  ftl 


(2.9) 


[X.-X*  II  <  llx,  -X*  I 

'  1  l,n"  —  "  l.n  l,n' 


1  A  X  1 

^ ,  m  -m ' 


m“n+l 


Let  0  denote  the  norm  of  the  one-step  ahead  prediction  error  vector,  that 
is  a»||  Xj-Xj^ll  ^“trCf-tr^Q.  It  follows  from  (2.8)  that 


t 


(2.10) 


a  -a<  11  X,-X,  11  -a<  II  I  A  X  11  . 

n  —  1  l.n"  —  *'  m  -m" 

ni*nrpl 

00 

Thus  the  rate  of  convergence  of  o  -a  is  determined  by  that  of  |1  A  A  X  1|. 

n  Ti  m  — m 

n+1 

And  the  rate  of  convergence  of  the  latter  is  determined  by  as  it  follows 

from  Theorem  3.1  (b).  The  next  section  la  devoted  to  the  study  of  the  rate  of 
convergence  of  q-ag/a^,  when  the  spectral  density  matrix  f  has  sufficiently 
smooth  entries.  We  note  that  for  a  univariate  process,  i.e.  q“l,  <1  -0^/0 

have  exactly  the  same  rate  of  convergence  (since  0^0^).  But,  the  situation  is 
completely  different  for  q>  1. 


3.  Rates  of  Convergence  of 

Since  (0^-^  ^  bounded  decreasing  sequence  it  follows  that  the  rate  of 

convergence  of  q0^-0Q  is  the  same  as  that  of  q-o^/o^.  The  next  theorem  which 
is  a  multivariate  extension  of  Theorem  1  in  Devinatz  [3]  plays  an  Important  role 
in  estimating  the  rate  of  convergence  of  5^^.  Parts  (b)  and  (c)  of  this  theorem 
is  used  in  finding  rates  of  convergence  for  the  linear  predictor  and  inter¬ 
polator  of  {Xjj}. 

Theorem  3. 1 


(a)  If  f”^e  ^qxq  matrix-valued  trigonometric  polynomial  of  degree 

n  with  H^YI,  0<Y<“,  then,  q-0^/0  <  X  ^/tr(f  ^-H)f(f  ^-H)*dm,  where  X>0 


is  a  constant. 


(b)  If  f>cl,  0<c«»,  then, 

q-aja  >c  y  trD  D*. 
O  n-  i*  mm 


m>n 


(c)  If  clff<dl,  0<cfd«»,  and  if  S  -  f  D.  e 

"  k-0 


ike 


with  trS  S*  <y«*>  for  all  n,  then 
n  n  — 


q-o./a  >f. .  7. y  tr  f  f*  ,  where  f  is  the  m*^^  Fourier 
^  0  n-''l+*'dY'  mm  m 


1 1 
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coefficient  f 

Proof .  (a)  Let  f“$$*,  H-P*P,  where  $  and  P  are  the  optimal  factors  of  f  and  H 

respectively,  cf.  [8],  [13,  p.  504].  Denoting  by  and  P^  the  constant  terms  of 


these  H  functions,  we  have 
qxq 


(3.1) 


/tr(f"^-H)f  (f“ -H)din  - /tr(<t*^4>"^-P*P)$<&*($*^<I>"^-P*P)dm 

=  /tr($“^P"^-$*P*)PP*(P*^$*^-P'J>)dm 
>Y/!!P4>-(**P*)"^||  e  dm. 


In  the  above  argument  we  have  used  the  fact  that  for  compatible  matrices  A,B 
trAB=trBA,  and  that  the  eigenvalues  of  (PP*) (0)  and  (P*P)(0)  are  the  same,  and 
they  are  greater  than  Y* 

Since  follows  that  P$-Pq^$q^  is  orthogonal  to  (<t'*P*)  ^-Pq^$q^. 


therefore. 


I,HS  of  (3.1)  >Y/l|Po^«0^-P*ll  gdm-Y/tr(I-$QPQP4)(*oPj*Q)"^I-$oPoP$)*dm,_ 


2.  v-1, 


>YX‘VtMoPQ^<^l^dm, 

2 

where  X  is  the  largest  eigenvalue  of  '^qPq^q*  ^^^d  the  desired  result  follows  from 
Lemma  2.1  (a),  (b). 

(b)  From  Corollary  2.2  we  have 

q-a_/a  ■/tr('I>  ^-^aZV  ^->^07  V  )*dm 

u  n  u  n  u  n 

>  c/ll  <I>  XJ  |1  ^dm 

E 

00 

>  c  y  tr  D  D*, 

m«n+l 

where  the  last  inequality  follows  from  the  matriclal  Parseval's  identity,  cf.  [151, 
P-  121]. 

(c)  By  the  matriclal  Parseval's  Identity  we  have 


|mf>n 


tr  f  f*  <  /||f“^-S*S  1|  I  dm<c‘Vll  \\  ^dm. 

m  m  n  n''  E  “  "  n  k 


n  n''  E 


iO 


Also, 

(|j.~L$*s*s  II  _<||  $“^-s  II  „+ll  s  -<i>*s*s  II 

"  n  n"  E— "  n"  E  "  n  n  n"  E 

and 

II  s  -$*S*S  II  J-tr(I-<i)*S*)S  S*(I-**S*)* 
"n  nn"£  nnn  n 

<(tr  S  S*)tr(4)"^-S  ) 

—  n  n  n  n 

<Ydl|  1. 

Thus, 

II  g<(l+/^)||  o'^-s^ll  £. 

and 

I,  « 

|m|>n 

by  Lemma  2. 1(b). 


Q.E.D. 

To  find  the  rate  of  convergence  of  entries  of  f  are  sufficiently 

smooth  we  need  to  have  a  matricial  extension  of  the  following  important  (univariate) 
theorem  due  to  Grenander  and  Rosenblatt  [4,5];  A  necessary  and  sufficient  condition 
that  least  exponentially  to  zero  as  n  tends  to  infinity, 

is  that  f(9)  coincides  in  [-'n',Tr]  almost  everywhere  with  a  function  which  is 
analytic  for  real  9  and  has  no  real  zeros. 

In  the  rest  of  this  section  we  prove  a  matricial  extension  of  this  theorem. 

Our  method  of  proof  is  similar  to  that  of  Devlnatz  [3].  However,  in  the  proof 
of  the  necessity  part  given  here,  we  do  not  use  the  properties  of  orthogonal 
polynomials  with  respect  to  f.  Instead,  we  use  a  lemma  due  to  Miamee  [9]  which 
provides  a  simpler  and  more  direct  proof  of  this  part.  In  this  section  by  the 
zeros  of  a  matrix-valued  function  f  we  mean  the  zeros  of  det  f. 
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The  following  theorem  provides  a  matrlclal  extension  of  the  sufficiency 
part  of  the  theorem  of  Grenander  and  Rosenblatt. 

Theorem  3.2.  Let  f^clt  0<c<<»,  with  entries  that  coincide  In  [-ir.Tr]  almost 
everywhere  with  functions  which  are  analytic  for  real  6, .  then 

6^-qo^-aQ»  0 (p“) ,  0<p< 1 . 

-1  ii 

Proof.  Since  f^cl.  It  follows  that  the  entries  of  f  -(f  ■*)  are  analytic  In 
an  open  annulus  {  Ze^;  p<|z|<~},  0^<1.  By  using  an  argument  similar  to  that 
In  [3,  pp.  114-115]  we  get  that  Fourier  coefficient  of  f^^  sat- 

Jr 

Isfles  f.  OiQ  )  and  that  S  ,  the  symmetric  partial  sum  of  the  Fourier  series 

1C  XI 

of  f  converges  uniformly  to  f  Thus,  we  have  from  Theorem  3.1  (a)  that 
for  n  large 


from  which  the  result  follows. 


Q . £ .D . 


By  using  an  argument  similar  to  that  In  the  proof  of  Theorem  3.2,  and 

Theorem  2  [14]  we  obtain  a  slower  rate  of  convergence  for  6^  under  much  weaker 

differentiability  requirement  on  the  entries  of  f.  It  Is  Interesting  to  compare 

this  result  with  Theorem  3.1  and  Its  special  case  In  Baxter  [1,  p.  138]. 

Theorem  3.3.  Let  f^cl,  0<c<®,  and  d  be  a  positive  Integer. 

-1 

(a)  If  all  the  entries  of  f  "(f  ■')  have  2d-th  derivative  on  [-tt.tt],  and  the 
Fourier  series  of  these  2d^th  derivatives  converge,  then 


(b) 


If  all  the  entries  of  f«(f.,)  have  2d-th  derivative  on  [-'ir,Tr],  and  the 
Fourier  series  of  these  2d-th  derivatives  are  absolutely  summable,  then 


■l. 
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Nextt  we  prove  a  multivariate  version  of  the  necessity  part  of  the  theorem 
of  Grenander  and  Rosenblatt.  For  a  matrix  define  a  norm  for  V 

by  Il'i'lli"  f\\  The  following  lemma  plays  an  important  role  in  our  proof 

of  this  result.  For  completeness  we  sketch  the  proof  of  this  leimna  [9]. 

Lemma  3.4  (Miamee).  If  L^(f)C  ^ben  there  exists  a  positive  constant  M 

such  that 

II  '‘'lliiM||'i'llL2(f)  *  fe’f  m^Cf). 

2  1 

Proof.  It  is  enough  to  show  that  the  operator  TtL  (^)'*’bqjjq  defined  by 
T(t).'i'  is  closed,  then  the  result  follows  from  the  closed  graph  theorem. 

Q • E  aD • 

The  assumption  L^(f)C  is  a  natural  one,  since  it  guarantees  that  the 

qxq 

Fourier  coefficients  of  any  function  in  the  spectral  domain  is  well-defined. 
Furthermore,  it  is  not  restrictive  as  the  following  simple  lemma  shows. 

Lemma  3.5.  L^(f)C  lJ^^  if  f"^e  lJ^^,  and  only  if  (det  f)‘2qeL^ 

The  next  theorem,  which  is  of  independent  interest,  shows  that  if 
goes  to  zero  sufficiently  rapidly,  then  the  entries  of  f  ^  have  summable  Fourier 
series  [3]. 

Theorem  3.6.  If  I  2"  6^  < »  and  L^(f)C  lJ^^,  then  the  entries  of  f"^  have 
n"l  2 

summable  Fourier  series. 

Proof.  We  have  from  Corollary  2.2,  Lemma  3.5  and  Cauchy-Schwartz  inequality  that 
(q-ao/an)’*-(/tr($'^-^5^  u„)f 

>(Mqr^  I  U^^(k)l  . 
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for  k>n+l,  where  U  ..  denotes  the  ij-th  element  of  U  . 

n 


Thus,  we  have 


»n+l 
2  ^ 


y  I  <  K  2  ^  •  n*l 

iC  ^  K»* 


lc-2“+l 


and  summing  both  sides  over  n  we  get 


00 

A  • 


i  ^  j“l»2» •  •  •  »q» 


which  implies  that  the  entries  of  i  have  summable  Fourier  series.  Hence,  the 
same  is  true  of  f  cf.  [7,  p.  31]. 

Q .  E .  D . 

Now  we  can  prove  the  multivariate  version  of  the  necessity  part  of  the  Grenander 
and  Rosenblatt's  theorem. 

Theorem  3.7.  If  6  0<p<l.  Then  f  has  no  real  zeros  and  all  the  entries  of 

u  _ 

f  are  analytic  on  [-ir,iT]. 

00 

Proof.  Since  T  6  <  <»,  it  follows  from  Theorem  3.6  that  f>cl  for  some  constant 

. .  ■ ti  ~ 

n«l 

0<c<®.  Thus,  from  Theorem  3.1  (b) 

00 

I  tr  D  D*  <  K  p", 
k-tri-1  * 

and  this  implies  that  the  entries  of  ^  and  are  analytic  functions  on 

[it, it].  Hence,  the  entries  of  f  ^  are  analytic.  Now,  f  ^  can  have  no  real 

zeros  since  this  would  contradict  the  fact  that  f  is  in  Thus,  f"(f  ^  is 

qxq 

analytic  with  no  real  zeros. 


Q.E.D. 


4.  Rates  of  Convergence  of  Finite  Linear  Predictors 


In  this  section  by  using  the  results  of  the  previous  section  we  find  rates  of 
coTtvergence  for  the  finite  linear  predictors  when  the  process  {  has  a  smooth 
spectral  density  function.  Here,  we  confine  our  attention  to  the  case  wl.cn  the 
entries  of  f  are  analytic  functions  of  6.  Similar  results  can  be  obtained  when 
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entries  of  f  have  properties  like  those  stated  In  Theorem  3.3. 

When  X]^  has  a  mean  convergent  autoregressive  representation  it  follows  from 
(2.8)  that  the  finite  linear  least  squares  predictor  of  X]^,  i.e.  Xj^  ^  converges 

y\ 

to  X^  in  the  norm  of  According  to  (2.9)  the  rate  of  convergence  in  this 

00  ^ 

case  Is  dominated  by  ||  T  AgPC-nH  and  ||  X,  -X*  ||  .  It  is  shown  by  Wiener 

n+1 

and  Masanl  [15  II]  that  the  boundedness  condition 


(4.1) 


cl  <  f  <  dl,  0  <  c  <  d  <  «, 


is  sufficient  for  the  existence  of  a  mean  convergent  autoregressive  representation 

A. 

of  x^.  For  weaker  conditions  and  more  up  to  date  results  of  this  type,  see 
[11,  12].  It  follows  from  (4.1)  that 

II  I  A|sX_,||  ^  1  d  /II  i  A.  e“®||  Jdm(e) 

n+1  n+l  “ 


(4.2) 


-  d  I  tr  A,  A| 

n+1 

OB 

<  d  o  y  tr  D  D* 

—  ■ 

n+1 


and  thus  from  Theorem  3.1  (b)  we  have 


(4.3) 


This  combined  with  Theorem  3.2  and  (2.8)  gives  the  following 
Theorem  4.1.  Let  { Xj^}  be  a  q-varlate  WSSP  with  the  spectral  density  matrix  f 
satisfying  (4.1)  whose  entries  are  analytic  function  of  6.  Then,  there  exists 
a  p,  0£p<l,  such  that 

(a)  IIV^i,„l|-  0(p"). 

(b)  0^-0  -  (9(p"). 

(c)  IIV*l,nll‘  ° 

(d)  tr  D^D*  -  0(p”)  and  tr  C*  -  0(p"). 
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•  /V  /V 

Next  we  concentrate  on  finding  the  rate  of  convergence  of  ||Xj^-X*  ^||  . 
Because  of  (2.9)  and  (4.3)  It  Is  enough  to  find  the  rate  of  convergence  of 
||  X*  ^||  .  For  q“l,  this  has  been  found  by  Baxter  [1]  using  a  certain 

Inequality  and  convergence  equivalence  Involving  Szego's  orthogonal  poly¬ 
nomials.  It  seems  very  difficult  to  obtain  matrlclal  extension  of  these 


results.  In  the  following  we  obtain  the  rate  of  convergence  of  ||Xj^  n~^l  n^^ 
when  l£^q<»,  without  using  the  notion  of  orthogonal  polynomials. 

A 

The  rate  of  convergence  of  ||  X*  ^|1  is  of  fundamental  Importance  in  the 

statistical  theory  of  multiple  time  series,  particularly  In  studying  the  asymptotic 
properties  of  the  autoregressive  estimator  of  the  spectrum,  cf.  [2,6].  This 
Importance  Is  revealed  by  the  following  simple  fact:  If  f^cl,  then 


(4.5) 


I  Ik  -C.  D  ll  •0(  11 X*  -X, 
'^nm  0  m"  ^  "  l,n  l,i 


11). 


which  provides  a  rate  of  convergence  to  zero  for  the  entries  of  the  matrices 
^nm"S  V  n-1,2,... 

A. 

In  the  following  we  find  the  rate  of  convergence  of  1)  X?  -X,  ||  by  using 

1  p  n  X  p  Ti ' 

a  slightly  different  but  equivalent  characterization  of  Xf  than  that  Introduced 

1  ,n 

and  used  In  Sections  2  and  3.  This  Is  based  on  Identifying  X*  as  the  unique 

i  t  n 

mlnlmlzer  of  the  matrlclal  functional  4)(Y)«(Xj-Y,  X^-Y),  YeL“8p{X_j^;  0<k<n-l}. 

We  say  that  Y_eL  is  a  mlnlmlzer  of  i[)(.)  and  write  ij;(Yrt)“min  t]'(Y)  if 
^  YeL 


t|<(Yo)<iKY),  for  all  YeL. 

It  Is  shown  In  [8,  p.  354]  that  such  a  Yq  exists  and  it  is  unique.  This 

unique  mlnlmlzer  Is  X*  Indeed.  Next  we  characterize  the  isomorph  of  X*  ^ 

in  the  spectral  domain.  Let 

(4.6)  G  "min  /PfP^dm, 

n 


P 


t 


t 
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th  “ 

where  the  min  is  taken  over  all  n  degree  matricial  polynomial  P(e)-  T  P  e' 

m 


ime 


with  Pq*I.  If  U  (0)*  I  V  is  the  minimizer  of  (A. 6),  then  V  can  be 

m-0  “ 

characterized  as  the  unique  n^^  degree  polynomial  with  V  “I  for  which 

nO 

fV  f  e  dm«0,  l<m<n. 
n  —  — 

A  different  and  more  useful  characterization  of  is  given  by  letting 

V  ^  U  and  showing  that 
n  n  n  “ 


iiP«0 


mini  (I-P4> )  (I-P-t )  *dm- 


(4.7) 


where  the  min  is  taken  over  all  n^^  degree  matricial  polynomials.  It  should  be 

noted  that  the  identities  in  (4.7)  are  essentially  the  matricial  version  of  the 

results  of  Lemma  2.1  and  Corollary  2.2.  Let  S  (0)«  T  D  e^"®  ,  with  these 

”  m-0  “ 

notations  and  the  fact  that  G  V  -C^D.-l  we  have 

n  nU  0  0 


J\  ”  II  I  (^  V  -C^D  )X  I 

l,n  l,n"  jjto  “  ®  ®  ”® 


We  have 


G-G<tT(G  -G).I-(0  -0)1, 
n  —  n  n 


Therefore, 

(4.9)  llG^-Gllg-o(a^-a). 
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The  race  of  convergence  of  obtained  as  follows: 

From  (4.7)  with  f^cl  we  have 

<^■11  W„ll  I  "W'’» 

m>n 

therefore, 

(4.10)  l  trD^D*)’‘+c'’*[tr(I-c\?;^ffS]**. 

m>n 

From  Theorem  3.1  (b)  we  have 

(4.11)  I  trD  D*  -  0(6  ), 

m>n  “  “  “ 

as  for  tr(I-oN?^^0^)  we  have  from  the  simple  Identity 

I-o\?‘VV’*(0  -<7)0"^<?**, 

n  n  n 

that 

trd-oV^oS-tiC'^C  -(?)  <  (trC‘2)**[tr(C„-«)^l**  <  kH  C? -gH  . 
n  tin—  n  n  —  njs 

Thus,  from  (4.9)  we  have 

(4.12)  trd-oV^O**)  -  0(0  -o). 

n  n 

By  combining  these  estimates  with  (2.9),  Theorems  3.2  and  4.1  (b)  we  get  the 
following 

Theorem  4.2.  Under  the  conditions  of  Theorem  4.1  we  have 

<•)  Z  E  -  •^IVo’nll 

m-0 

<"> 

(c)  llXi-Xf^„ll  -  0(p"). 
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Theorem  3.1  (c)  can  be  used  to  find  the  rate  of  convergence  for  the  finite 
linear  interpolator  of  {  X^} .  For  this  let  us  assume  that  all  the  values  of  {  X^} 
are  known,  except  for  the  value  Xq.  Then,  under  the  assumption  that  f  £^qxq* 
it  is  known  [10]  that  the  linear  least  squares  interpolator  of  X^,  denoted  by 


Xq,  can  formally  be  written  as 


(4.13) 


w  ''-I  r  ^ 

"o'-'o  .LU\ 


where  f,  is  the  k*"^  Fourier  coefficient  of  the  matrix  f~^.  Sufficient  conditions 
k 

for  the  mean  convergence  of  the  series  in  (4.13)  are  given  in  [10].  In  part¬ 
icular,  it  follows  from  Theorem  7  [10]  that  the  boundedness  condition  (4.1) 
guarantees  the  convergence  of  this  series. 

Now  one  can  form  a  finite  Interpolator  for  Xq,  denoted  by  Xq  ^  and  defined 

by  Xq  1  “O’™  of  this  approximation  is  given  by 

*  k*-n 

k^O 


'-1  II  2  r 

V  1^  Vk* 

0  |kl>n 


Thus,  from  Theorem  3.1  (c)  we  have 


and  combining  this  with  Theorem  3.2  we  get  the  following 
Theorem  4.3.  Under  the  conditions  of  Theorem  4.1  we  have 


lV^0,nll  "  0<P<1. 
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An  exponential  rate  of  convergence  for  the  finite  linear  Interpolator  of 
a  univariate  stationary  process,  when  its  past  and  future  are  at  positive  angle 
[10,11],  is  obtained  by  Salehi  [13].  However,  in  his  work  the  finite  linear 
interpolator  is  obtained  by  using  the  Von  Neumann's  alternating  projections, 
as  a  result  it  is  less  feasible  for  computation  since  it  involves  the  one¬ 
sided  Innovation  process  the  coefficients  of  the  optimal  factor  of  the 

density.  An  interesting  by  product  of  Salehi* s  approach  is  the  identification 
of  p  in  Theorem  4.3  with  the  cosine  of  the  angle  between  the  past  and  future 
of  the  process  In  view  of  this,  one  may  raise  the  following  natural  and 

useful  question:  Is  it  possible  to  identify  the  constant  P  appearing  in  this 
paper  (say,  in  Theorem  3.2)  with  the  cosine  of  the  angle  between  the  past  and 

future  of  the  process  {X  }?  If  the  answer  is  positive  then  in  an  obvious  manner, 

n 

the  results  of  this  paper  confirm  the  intuitively  appealing  fact  that  the  closer 
this  angle  is  to  ir/2  the  faster  the  finite  predictors  will  converge  to  the 

A 

infinite  predictor  X^^. 

Another  problem  which  is  worthy  of  study  in  connection  with  this  work  is 
the  Infinite-dimensional  generalization  of  the  results  presented  here.  Presence 
of  q  in  Theorem  3.1  and  5^  shows  that  the  method  of  proof  used  here  is  intrinsically 
finite-dimensional.  It  is  of  Interest  to  know,  e.g.  the  analogues  of  Theorem  3.1 
and  6  in  the  infinite  dimensional  case. 
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